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Executive  Summary 

Title:  Building  Potential  for  Pilot  Expertise:  Can  Understanding  How  People  Think  and 
Make  Decisions  Improve  the  Ability  of  Military  Flight  Training  to  Create  Potential  Pilot 
Expertise? 

Author:  LCDR  Douglas  M.  Tempest,  United  States  Navy 

Thesis:  Based  on  current  understanding  of  the  naturalistic  decision  making  process, 

Navy  flight  schools  can  improve  the  flight  training  program  and  increase  the  opportunity 
to  create  and  train  future  expert  pilots. 

Discussion:  Naturalistic  decision-making  is  the  study  of  how  people  use  their  experience 
to  make  decisions  in  field  settings.  Understanding  that  this  is  actually  how  people  think 
and  make  decisions  is  at  the  root  of  how  to  teach  them  to  improve  this  skill. 

The  flight-training  program  is  designed  to  teach  basic  tactics,  techniques,  and  procedures 
so  that  an  inexperienced  pilot  can  safely  operate  a  complex  aircraft.  At  the  completion  of 
flight  training  the  new  pilot  has  enough  basic  knowledge  and  skill  that  he  can  safely 
perform  the  basic  maneuvers  in  the  aircraft.  The  expectation  is  that  this  level  of  training 
is  good  enough  until  these  new  pilots  can  build  up  enough  experience  to  become  expert 
pilots,  a  process  requiring  many  hours. 

Flight  school  is,  therefore,  a  safety  stopgap  that  enables  the  truly  inexperienced  student  to 
learn  just  enough  to  live  to  possibly  become  an  expert.  The  problem  with  training  merely 
competent  pilots  is  that  this  goal  stops  well  short  of  what  is  truly  needed.  The  Navy 
needs  expert  pilots  who  can  react  quickly  and  intuitively  in  complex  situations  to  execute 
sound  judgment.  Producing  expert  pilots  should  be  the  true  end  goal  of  the  flight-training 
program.  This  goal  is  not  reached  nor  sought  during  flight  training  but  instead  may  occur 
years  later  after  the  novice  pilots  have  built  up  enough  experience. 

Conclusion:  By  understanding  how  experts  make  decisions,  through  the  use  of  a 
technique  known  as  Cognitive  Task  Analysis  (CTA),  and  focusing  training  efforts  on 
building  experience  through  the  lens  of  expertise,  Naval  flight  training  can  improve  the 
pattern-recognition  and  intuitive  decision-making  capability  of  novice  pilots  and 
potentially  bridge  the  gap  between  flight  training  and  expertise. 
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“The  secret  to  being  a  good  pilot  is  good  judgment .  .  . 
the  art  of  teaching  judgment  to  a  pilot  is  a  secret.” 

~  Dr.  Fritz  H.  Brecke1 

Preface:  A  Pump,  Not  a  Filter 

In  the  summer  of  2010,  when  I  left  flight  school  for  the  3rd  time,  I  was 
transitioning  to  my  first  non-flying  tour  and  realized  that  I  had  spent  almost  ten  years 
flying  and  over  half  that  time  was  spent  in  flight  training  in  one  capacity  or  another.  I 
began  my  naval  aviation  career  with  no  flight  experience  and  went  through  the  standard 
process  to  become  a  pilot.  Later,  when  the  helicopter  I  flew  was  retired,  I  reentered  flight 
school  as  an  experienced  junior  pilot  transitioning  to  a  new  aircraft.  After  another  short 
operational  tour,  I  returned  to  flight  school,  this  time,  as  an  instructor. 

Throughout  all  my  learning  and  teaching  experiences,  I  was  continually  frustrated 
by  the  ineffectiveness  and  inefficiencies  within  the  program.  As  an  inexperienced 
student,  I  spent  most  of  my  time  studying  and  memorizing  the  complex  engineering  of 
the  aircraft.  I  also  spent  time  memorizing  procedures,  but  the  ability  to  perfonn  those 
procedures  safely  was  only  addressed  during  the  instructional  flights.  Later,  as  a  more 
experienced  junior  pilot  transitioning  to  a  new  aircraft,  references  to  the  previous  aircraft 
became  “dirty  words”  to  the  instructor  pilots.  They  didn’t  know  anything  about  the  other 
aircraft  and  didn’t  want  to  hear  about  how  things  used  to  be  done.  When  I  finally 
returned  as  an  instructor,  I  quickly  realized  that  many  instructors  were  extremely  skilled 
at  operating  the  aircraft,  but  they  were  ignorant  of  the  specific  actions  they  took  to 
perform  at  that  level.  Intuitively,  they  knew  how  to  fly  and  did  so  well,  but  they  had 
trouble  explaining  what  they  did.  I  spent  the  next  three  years  trying  to  break  down  each 
maneuver  to  try  to  determine  the  best  way  to  explain  it  to  inexperienced  students.  I  tried 
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to  determine  what  the  other  instructors  and  I  were  doing  differently  than  an  inexperienced 
pilot,  what  we  were  looking  at  differently  and  what  we  were  thinking  about  differently. 
My  hope  was  that  I  could  find  more  efficient  ways  to  teach  the  basic  maneuvers  to  help 
students  understand  them  more  quickly  instead  of  just  repeating  the  maneuver  enough 
times  that  the  student  eventually  figured  out  how  to  do  it  on  their  own. 

My  efforts  to  teach  students  to  fly  prompted  me  to  think  about  what  we  were 
teaching  them.  We  were  teaching  the  basic  tactics,  techniques,  and  procedures  to  help 
them  learn  to  operate  an  aircraft  safely  at  the  most  basic  level.  As  instructors,  we  might 
question  a  student’s  judgment  and  whether  he  would  have  a  successful  career  in  the 
Navy,  but  we  were  in  the  business  of  producing  pilots  with  only  a  very  basic  set  of  skills, 
not  evaluating  or  teaching  their  higher  level  functioning  and  decision  making  ability. 

This  concept  led  to  a  fellow  instructor  jokingly  reminding  us  that  flight  school  is  “a 
pump,  not  a  filter.”  Sometimes  we  would  even  call  someone  we  knew  at  an  operational 
squadron  to  warn  them  about  an  individual  student  heading  their  way  after  we  had  signed 
off  on  his  certification.  We  were  so  concerned  about  their  ability  to  function  and  make 
sound  decisions  that  we  wanted  to  warn  our  friends  of  the  risks  we  saw,  but  we  did  not 
feel  that  we  had  grounds  to  remove  them  from  aviation.  We  passed  them  on  and  left 
them  for  others  to  deal  with  at  a  level  when  their  judgment  would  actually  be  a  criterion 
for  evaluation.  Our  job  was  to  train  them  in  the  basics  not  evaluate  their  higher  level 
functioning.  This  mindset  always  perturbed  me.  Why  were  we  not  using  the  more 
experienced  aviators  filling  instructor  billets  to  provide  insight?  What  was  the  real  goal 
of  flight  school  and  were  we  achieving  that  goal? 
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My  belief  is  that  the  structure  of  flight  school  has  been  designed  to  teach  the 
basics  required  to  keep  new  aviators  alive  long  enough  for  them  to  build  the  experience, 
in  the  Fleet,  necessary  to  become  effective  military  pilots.  By  reducing  the  requirements 
to  such  an  extreme,  we  make  the  new  pilots  nothing  more  than  human  autopilots.  We  fail 
to  address  the  need  for  sound  judgment  in  the  fast  paced  and  dynamic  world  of  naval 
aviation.  We  should  be  training  air  warriors  not  autopilots  and  advancement  in  the 
understanding  of  the  human  mind  and  the  decision  making  process,  especially  the  realm 
of  intuition,  will  provide  potential  methods  to  advance  the  flight  training. 
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The  Current  Frame  of  Reference:  The  Way  Things  Are 

In  order  to  recognize  the  need  for  change,  a  brief  summary  of  the  current  flight 
school  system  is  necessary.  This  summary  will  focus  on  the  relevant  issues  for 
discussion,  such  as  how  students  are  taught  and  the  types  of  material  covered,  rather  than 
details  of  the  procedures  and  policies  beyond  the  scope  of  this  discussion. 

Naval  flight  training  consists  of  a  series  of  phases.  The  first  three  phases, 
introductory,  intermediate,  and  advanced,  are  typically  known  as  “flight  school”.  At  the 
completion  of  advanced  training,  the  student  has  completed  an  introductory  course  in  a 
helicopter,  multi-engine  airplane,  or  a  jet.  At  the  completion  of  this  phase,  the  student 
earns  his  wings,  the  military  designation  qualifying  him  as  a  pilot  instead  of  a  student 
pilot.  While  the  fonner  students  may  be  designated  pilots,  they  are  still  learning.  From 
the  advanced  course,  the  new  aviators  move  on  to  a  fleet  replacement  squadron  (FRS). 
The  FRS  is  where  the  new  aviators  are  first  introduced  to  a  military  aircraft  and  taught 
rudimentary  tactics. 

The  standard  flow  of  events  within  any  phase  is  a  series  of  classes,  known  as  a 
ground  school,  followed  by  a  series  of  events  in  a  military  simulator  and  then  a  period  of 
flight  instruction  in  an  actual  aircraft.  The  flow  is  repeated  for  each  separate  mission  or 
tactical  category.  The  ground  schools  typically  consist  of  an  instructor-guided  review  of 
material  learned  initially  through  self-study.  The  instructor  teaches  the  class  using  a 
computer  aided  lesson  plan,  and  the  class  concludes  with  a  written  exam.  Classes  are 
taught  on  tactics,  techniques,  and  procedures  as  well  as  the  engineering  components  on 
all  the  aircraft  systems.  Most  of  the  engineering  classes  last  two  hours,  but  the  tactics 
classes  typically  consist  of  multiple  lessons  and  last  two  days.  The  simulator  and  flight 
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events  each  consist  of  an  event  brief  followed  by  the  event  itself.  The  briefs  for  the 
simulator  typically  last  thirty  minutes  and  set  up  what  will  be  covered  with  the  simulator. 
The  simulator  teaches  procedures  and  techniques  utilizing  real  aircraft  controls  without 
the  risk  of  mistakes  of  an  actual  flight.  The  flight  events  consist  of  a  brief  lasting  two  to 
three  hours  followed  by  a  flight  with  an  instructor.  The  brief  is  effectively  an  oral  exam 
covering  one  of  the  aircraft  systems  as  well  as  some  tactics,  techniques,  and  procedures 
relevant  to  the  flight. 

Each  flight  has  a  list  of  maneuvers  to  be  perfonned  on  the  flight.  Each  maneuver 
is  designated  as  “demonstrate,  introduce,  or  review.”  “Demonstrate”  means  that  the 
instructor  performs  the  maneuver  while  the  student  observes.  “Introduce”  means  that  the 
student  performs  the  maneuver  with  coaching  from  the  instructor.  “Review”  means  that 
the  student  is  expected  to  perform  the  maneuver  and  is  graded  on  his  ability  to  perfonn. 
When  the  student  completes  the  prescribed  training  syllabus,  he  leaves  the  FRS  as  a  pilot 
qualified  in  model  (PQM).  This  designation  means  that  the  pilot  can  safely  perform  the 
basic  maneuvers  of  that  model  of  aircraft.  This  designation  represents  the  student’s 
departure  from  the  formal  training  commands,  but  his  training  is  still  not  complete. 

To  utilize  the  helicopter  community  as  an  example,  the  PQM  has  additional 
qualifications  to  earn.  The  next  step  is  to  be  designated  as  a  helicopter  second  pilot 
(H2P)  indicating  enough  experience  and  proficiency  to  be  a  competent  copilot  during 
operational  missions.  The  next  designation  is  as  a  helicopter  aircraft  commander  (HAC). 
A  HAC  is  a  helicopter  pilot  qualified  to  fly  with  and  train  PQMs  and  H2Ps.  The  HAC  is 
the  pilot  in  command  during  missions.  Some  missions  do  require  additional  training  and 
designations,  but  the  HAC  is  the  first  level  where  a  pilot  is  authorized  to  take  the  aircraft 
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out  on  his  or  her  own  and  is  responsible  for  the  safe  completion  of  the  mission  as  well  as 
the  lives  of  the  crew.  The  top  aircraft  commanders  within  a  squadron  are  usually  chosen 
to  return  to  the  FRS  or  to  one  of  the  earlier  stages  of  flight  school  to  become  instructors. 

Introduction:  Thinking  about  Thinking 

The  flight-training  program  reviewed  above  is  designed  to  teach  basic  tactics, 
techniques,  and  procedures  so  that  an  inexperienced  pilot  can  safely  operate  a  complex 
aircraft.  There  are  very  sound  reasons  for  this  program  to  exist,  and  there  will  probably 
be  some  reluctance  to  change  a  program  that  has  actually  demonstrated  a  reduction  in 
aviation  mishaps,  so  the  question  becomes  why  change  anything? 

If  the  end  goal  of  flight  school  is  simply  a  pilot  that  is  safe  to  perfonn  basic 
maneuvers  with  no  judgment  required,  then  there  is  no  reason  to  change,  but  I  do  not 
believe  this  to  be  the  case.  At  the  completion  of  flight  training  the  new  pilot  has  enough 
basic  knowledge  and  skill  that  he  can  safely  perform  the  basic  maneuvers  in  the  aircraft. 
The  expectation  is  that  this  level  of  training  is  good  enough  until  these  new  pilots  can 
build  up  enough  experience  to  become  expert  pilots,  a  process  requiring  many  hours. 
Flight  school  is,  therefore,  a  safety  stopgap  that  enables  the  truly  inexperienced  student  to 
learn  just  enough  to  live  to  possibly  become  an  expert.  The  problem  with  training  merely 
competent  pilots  is  that  this  goal  stops  well  short  of  what  is  truly  needed.  The  Navy  does 
not  need  competent  pilots.  The  Navy  needs  expert  pilots  who  can  react  quickly  and 
intuitively  in  complex  situations  to  execute  sound  judgment.  Producing  expert  pilots 
should  be  the  true  end  goal  of  the  flight-training  program.  This  goal  is  not  reached  nor 
sought  during  flight  training  but  instead  may  occur  years  later  after  the  novice  pilots  have 
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built  up  enough  experience.  Based  on  current  understanding  of  the  naturalistic  decision 
making  process.  Navy  flight  schools  can  improve  the  flight  training  program  and  increase 
the  opportunity  to  create  and  train  future  expert  pilots.  The  primary  expectation  is  that  a 
shift  in  training  methodology  will  bridge  the  gap  between  the  end  of  traditional  flight 
school  and  when  a  pilot  becomes  an  expert. 

Research  (or  the  Lack  Thereof) 

Although  research  has  been  done  on  ways  to  utilize  simulators  more  effectively  in 
flight  training,  there  is  no  indication  of  research  utilizing  modem  understanding  of  how 
people  think  to  determine  more  effective  means  of  training.  This  paper  tries  to  bridge  the 
theoretical  gap  that  exists  between  the  current  understanding  of  how  people  think  and 
make  decisions  and  the  practical  implementation  of  tools  to  improve  training  techniques. 
Some  of  these  tools  include  Cognitive  Task  Analysis  (CTA)  and  a  video  game  called 
Space  Fortress.  There  may  be  more  and  better  tools  available,  but  these  are  examples  of 
some  of  the  ways  to  apply  the  theory  of  how  people  think,  leam,  and  decide  to  modern 
flight  training.  The  theory  behind  CTA  is  that  training  people  how  to  think  is  much 
better  than  teaching  them  what  to  think. 

This  study  uses  examples;  most  come  from  the  research  that  created  the  theories 
described  and  are,  therefore,  not  specific  to  aviation.  Where  applicable  and  available,  I 
have  tried  to  include  personal  experiences  and  examples  specific  to  aviation  to  illustrate 
how  the  theory  can  apply.  Ultimately  the  Navy  can  synthesize  various  complex  theories 
into  a  cohesive  whole  that  can  be  applied  to  improve  military  flight  training. 
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Creating  Expertise:  Nature  versus  Nurture 

An  expert  must  meet  three  criteria.  First,  an  expert’s  perfonnance  is  consistently 
superior  to  that  of  his  peers.  Second,  expertise  produces  concrete  results.  Third,  true 
expertise  can  be  measured  and  replicated,  such  as  in  a  lab  setting.-  Appendix  A  contains 
a  more  thorough  discussion  of  the  many  advantages  of  expertise.  In  1985  Professor 
Benjamin  Bloom  published  his  book,  Developing  Talent  in  Young  People,  and  examined 
the  evolution  of  talent  in  individuals.  He  discovered  that  there  were  no  innate  indicators 
of  future  success,  with  the  exception  of  height  and  body  size  in  fields  where  physicality 
matters,  such  as  sports.  Instead,  he  discovered  that  there  were  three  things  that 
determined  superior  performance.  The  three  indicators  were  intense  practice,  study  with 
a  dedicated  teacher,  and  an  enthusiastic  and  supportive  family.  Measurable  intelligence 
seems  to  play  little  role  in  future  success,  and  the  most  important  of  these  three  indicators 
was  the  amount  and  quality  of  the  practice.  “Consistently  and  overwhelmingly,  the 
evidence  showed  that  experts  are  always  made,  not  born.”  The  implications  of  this 
simple  finding  are  profound.  If  the  key  to  expertise  lies  in  the  hands  of  the  individual  and 
does  not  rely  on  fate  or  genetics,  then  expertise  can  be  taught.  This  is  not  to  imply  that 
creating  experts  is  easy;  the  experience  necessary  still  takes  years  of  dedicated  effort, 
usually  around  ten  years  or  10,000  hours  of  experience,  but  it  does  not  rely  on  chance. 

The  current  flight  school  system  expects  novice  pilots  to  gain  the  “right”  experience 
through  some  form  of  osmosis  from  those  with  more  experience.  Flight  training  fails  to 
ensure  that  the  desired  experiences  are  created. 

Each  operational  squadron  creates  a  syllabus  through  which  novice  pilots  progress 
and  qualify  as  aircraft  commanders.  These  syllabi  typically  consist  of  a  number  of 
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mission  areas  that  the  novice  must  experience  and  get  a  grade  card  signed  off  by  an 
aircraft  commander.  The  cards  specify  what  missions  are  required  as  well  as  some 
specific  areas  that  must  be  addressed.  Procedures  must  be  tested,  but  the  training 
commands  consider  the  specific  thought  process  of  the  more  experienced  pilot  as  just  a 
technique  and  advisory  only.  To  give  a  personal  anecdote,  while  deployed  with  my 
second  helicopter,  I  was  training  my  second  pilots  on  daytime  search  and  rescue  (SAR) 
techniques.  My  copilot  thought  he  knew  everything  he  needed  to  know  about  SAR 
because  he’d  learned  all  the  procedures  during  training.  After  I  had  put  him  through  his 
paces,  he  was  finally  able  to  rescue  the  simulated  survivor.  After  he  took  a  big  sigh  of 
relief,  I  asked  him  what  we  were  going  to  do  next.  He  replied  that  we  were  going  to  take 
the  survivor  back  to  the  ship  for  medical  treatment.  That’s  when  he  realized  he  didn’t 
know  where  the  ship  was  anymore.  He’d  lost  sight  of  it  while  he  was  working  on  the 
SAR  and  the  ship  had  decided  to  turn  off  its  navigational  aid  while  we  were  away.  He 
was  operating  in  the  middle  of  the  ocean  with  no  land  in  sight  and  no  idea  how  to  get  the 
aircraft  back  to  safety.  I  was  able  to  use  his  mistakes  to  highlight  how  I  had  thought 
about  the  problem  and  what  I’d  done  to  maintain  situational  awareness  of  the  ship’s 
location  as  well  as  the  nearest  safe  point  of  land.  What  I  taught  my  copilot  were  my 
techniques  for  that  scenario,  but  there  is  no  standardization  to  ensure  that  the  next  aircraft 
commander  would  challenge  him  to  explain  his  thought  process.  The  requirement  is  only 
that  the  novice  pilots  are  exposed  to  different  missions  and  procedures.  The  assumption 
is  that  they  will  figure  out  how  to  think  about  these  situations. 

Deliberate  Practice 
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Practice  is  the  most  valuable  variable  in  terms  of  creating  expertise  as  well  as  the 
most  controllable  aspect.  The  term  practice  can  be  confusing,  as  it  implies  that  any  time 
spent  working  on  a  subject  eventually  grants  expertise,  but  this  is  not  the  case.  Not  all 
practice  makes  perfect,  and  most  people  practice  by  working  on  things  they  already  know 
how  to  do.  Practice  alone  is  not  enough;  it  must  be  a  specific  type  of  practice.  K.  Anders 
Ericsson,  Michael  J.  Prietula,  and  Edward  T.  Cokely  in  their  article  “The  Making  of  an 
Expert”  refer  to  this  type  of  practice  as  deliberate  practice.  Geoff  Colvin  further  develops 
the  term  in  his  book  Talent  is  Overrated.  Deliberate  practice  entails  considerable, 
specific,  and  sustained  efforts  to  do  something  you  cannot  initially  do  well  if  at  all.4 
Deliberate  practice  is  designed  specifically  to  improve  performance,  often  with  a 
teacher’s  help.  It  needs  to  be  repeated  a  lot;  feedback  on  results  must  be  given 
continuously;  it  is  highly  demanding  mentally,  and  it  is  not  much  fun.5 

The  specific  design  aspect  of  deliberate  practice  requires  a  plan  or  design  to 
execute.  Most  fields  of  endeavor  have  bodies  of  knowledge  on  how  performance  is 
developed  and  improved.  An  amateur  simply  trying  things  out  or  working  on  skills  that 
he  already  knows  is  not  drawing  on  this  knowledge  and  his  practice  is  not  designed  to 
utilize  this  knowledge  to  improve  performance.  Improvement  without  design  is  simply 
accidental.  Dedicated  practice  requires  purposeful  training,  not  haphazard  practice  time. 
Usually,  a  teacher  or  coach  can  drastically  improve  the  ability  to  deliberately  practice. 
Teachers  not  only  have  a  vast  knowledge  of  the  techniques  needed  to  improve,  they  also 
are  unbiased  observers  capable  of  determining  the  best  practice  activity  to  improve 
performance.  One  way  of  describing  this  process  is  to  think  of  three  concentric  circles. 
The  inner  circle  is  the  comfort  zone.  The  middle  circle  is  the  learning  zone,  and  the  outer 
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circle  is  the  panic  zone.  The  comfort  zone  contains  activities  that  are  already  mastered 
and  the  panic  zone  contains  activities  that  are  still  out  of  reach  and  in  which  the  student  is 
completely  helpless.  Progress  occurs  when  activities  are  chosen  in  the  neither  too  simple 
nor  too  demanding  learning  zone.  Only  the  learning  zone  contains  activities  that  stretch 
the  limits  of  one’s  skills  and  it  is  through  this  stretching  process  that  improvement  occurs. 
In  addition  to  being  at  the  correct  level  of  difficulty,  the  activities  need  to  be  repeatable. 
High  volume  of  repetition  is  key  for  building  up  new  skills  as  well  as  improving  on  old 
ones.  Repetition  allows  the  building  of  consistency  and  eventually  will  imprint  the  feel 
of  when  the  technique  is  used  correctly.6 

Continuous  feedback  is  also  necessary  for  deliberate  practice  because  it  is  the 
only  way  to  detennine  if  the  training  is  working.  In  sports,  seeing  the  results  of  practice 
is  very  straightforward.  For  example,  basketball  players  know  very  quickly  if  they  make 
their  shot.  The  feedback  makes  for  immediate  adjustments  and  corrections.  Feedback 
continues  in  an  endless  cycle  with  the  adjustments  it  requires.  This  process  requires  a 
fairly  high  level  of  concentration  and  focus  that  puts  a  high  demand  on  even  a  mainly 
physical  deliberate  practice.  This  level  of  concentration  is  exhausting,  and  no  one  can 
sustain  it  for  long  periods  of  time.  Four  or  five  hours  of  deliberate  practice  seem  to  be 
the  upper  limit,  and  only  the  masters  in  a  field  reach  this  level.  Two  or  less  hours  appears 
to  be  a  much  more  consistent  average  sustainable  deliberate  practice  time.  This  extreme 
level  of  concentration  is  exhausting  and  not  much  fun. 7 

Expert  Pilot? 

Since  there  are  significant  benefits  to  being  an  expert,  and  expertise  can  be  taught, 
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the  next  step  is  to  detennine  what  makes  an  expert  pilot  and  how  can  these  skills  can  be 
taught.  While  “stick  and  rudder”  skills  are  invaluable,  they  are  actually  rudimentary.  To 
an  expert  pilot,  they  are  second  nature.  The  expert  pilot  performs  the  manipulation 
without  thought  and  knowing  where  to  position  the  aircraft  in  the  sky  becomes  more 
relevant  than  how  to  get  the  aircraft  into  that  position.  The  experiments  of  psychologist 
Daniel  Gopher  support  this  claim;  Gopher  had  Israeli  Air  Force  cadets  play  a  video  game 
called  Space  Fortress.  The  game  exercised  memory  and  attention  by  requiring  players  to 
maneuver  a  ship  through  a  frictionless,  hostile  environment  while  firing  missiles  to  defeat 
an  enemy  and  avoid  being  shot.  In  one  hour  a  day  for  ten  days  the  cadets  playing  the 
game  showed  an  almost  thirty  percent  improvement  in  actual  flight  performance 

o 

compared  to  the  cadets  who  did  not  play.  The  video  game  did  not  teach  real  world 
techniques  or  procedures  but  instead  challenged  the  players  to  improve  memory  and 
attention.  The  improvement  in  memory  and  attention  over  ten  hours  of  play  resulted 
directly  in  an  improvement  of  thirty  percent  in  actual  flight  training.  Attention  and 
memory  help  determine  where  to  position  the  aircraft  based  on  the  environment  and 
situation  and  are  part  of  sound  judgment.  Therefore,  sound  judgment,  not  basic  flying 
skills,  is  more  relevant  to  expert  pilot.  Judgment  is  the  “ability  to  judge,  make  a  decision, 
or  form  an  opinion  objectively,  authoritatively,  and  wisely,  especially  in  matters  affecting 
action;  good  sense;  discretion.”9  More  colloquially,  judgment  is  the  ability  to  make  the 
“right”  decision  under  conditions  of  uncertainty.  The  difficulty  of  judgment  is 
determined  by  the  time  constraint,  uncertainty,  cognitive  complexity,  and  stress  of  the 
situation.10  To  exercise  good  judgment,  an  actor  must  overcome  these  four  difficulties. 
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Decision-Making 

To  determine  how  to  train  someone  to  make  good  decisions,  instructors  must  first 
know  how  people  make  decisions.  The  traditional  view  is  that  decision-making  is  done 
through  rational  analysis.  The  implication  is  that  situations  are  analyzed  and 
decontextualized  to  grasp  the  situation  and  allow  a  trainee  to  calculate  the  best  solution. 
The  traditional  view  was  thought  to  apply  even  in  time-critical  decision-making. 

Rational  actors  quickly  weigh  options  and  decide.  Modern  studies  of  naturalistic 
decision-making  are  finding  that  outside  the  laboratory  or  the  office  the  traditional  theory 
rarely  applies.  In  Klein’s  opinion,  only  specific  areas,  such  as  scientific  or  technological 
research,  are  appropriate  for  analytical  decision-making.  Rational  thinking  has  limited 
applicability  in  most  decisions  because  the  rules  are  ambiguous  and  calculations  difficult 
or  impossible. 1 1 

To  illustrate  this  within  aviation,  every  time  a  pilot  lands  an  aircraft,  there  are 
infinite  variations  on  speed  and  glideslope  available.  Based  on  the  current  environmental 
conditions,  there  may  or  may  not  be  one  “best”  solution;  the  one  decision  for  the  pilot  to 
find.  Instead  there  are  usually  multiple  solutions  that  satisfy  the  conditions  of  a  safe 
landing.  A  pilot  who  attempted  to  find  the  perfect  landing  would  run  out  of  gas  before  a 
solution  was  found.  By  contrast  a  pilot  who  accepted  satisfactory  solutions  would  be 
able  to  practice  multiple  landings  in  the  same  time,  each  time  gaining  experience  and 
building  pattern  recognition.  This  may  seem  obvious  and  the  idea  of  a  pilot  running  out 
of  gas  while  searching  for  a  perfect  landing  may  seem  far-fetched,  but  that  is  actually  the 
point.  People  use  experience  and  intuition  to  make  decisions  a  majority  of  the  time. 
Naturalistic  decision-making  is  the  study  of  how  people  use  their  experience  to  make 
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decisions  in  field  settings.  Understanding  that  this  is  actually  how  people  think  and  make 
decisions  is  at  the  root  of  how  to  teach  them  to  improve  this  skill. 

Systems  theory  as  well  as  complexity  theory  teaches  that  complex  adaptive 
systems  exhibit  a  phenomenon  known  as  emergence  where  the  sum  is  different  or 
possibly  greater  than  the  sum  of  its  parts.  Robert  Jervis  in  his  book  System  Effects  gives  a 
simple  example  of  this  relationship  using  a  piano.  Each  note,  when  struck  independently, 
has  a  predictable  nature,  but  when  two  notes  are  struck  together,  they  create  a  chord  with 
properties  that  are  different  from  either  note  taken  separately.  The  chord  has  a  new 
attribute  that  no  individual  component  had.  Analysis  cannot  break  these  systems  down 
to  find  solutions;  they  are  unpredictable,  but  despite  their  unpredictable  nature,  they  are 
bounded  and  patterns  can  emerge. 

Pattern  Recognition 

Pattern  recognition  is  a  quality  of  an  expert  and  develops  from  experience, 
especially  experience  enhanced  by  deliberate  practice.  Pattern  recognition  and  mental 
simulation  are  also  the  two  pillars  of  naturalistic  decision-making.  Gary  Klein 
discovered  in  his  book  Sources  of  Power  that  most  decision  makers,  especially  in  high 
risk,  time-critical  positions,  do  not  use  analytical  decision-making.  In  fact,  when  asked, 
most  people  did  not  even  think  they  had  made  a  decision  at  all.  The  primary  study  was  of 
fire  battalion  commanders,  making  life  and  death  decisions  in  the  midst  of  fighting  a  fire. 
These  commanders  saw  the  situation,  recognized  a  pattern  from  their  experiences  and 
could  use  that  pattern  to  create  mental  simulations  of  what  they  should  do  next  based  on 
what  they  thought  was  happening.  Since  these  actions  seemed  obvious  and  no  other 
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options  were  considered,  the  commanders  never  recognized  a  decision  point  in  the 
thought  process.  Figure  1  in  Appendix  B  is  a  perceptual  tool  to  illustrate  Gary  Klein’s 
view  of  what  sources  of  power  enable  people  to  make  decisions. 

Intuition 

“ Intuition  depends  on  the  use  of  experience  to  recognize  key  patterns  that  indicate 
the  dynamics  of  the  situation.  Because  patterns  can  be  subtle,  people  often  cannot 

1  T 

describe  what  they  noticed,  or  how  they  judged  a  situation  as  typical  or  atypical.” 
Intuition  creates  an  emotional  reaction  to  anticipated  consequences  of  good  or  bad 
choices  well  before  an  individual  consciously  recognizes  the  options.  One  of  the 
firefighter  commanders  studied  by  Klein  was  convinced  he  had  ESP.  He  had  evacuated  a 
building  seconds  before  the  floor  had  collapsed  and  was  convinced  he  had  a  sixth  sense 
that  had  warned  him.  What  Klein  discovered  was  that  the  commander  had 
subconsciously  recognized  minor  details  of  the  situation  that  stood  out  as  atypical.  He 
intuitively  realized  the  fire  had  started  in  a  different  location  from  that  originally  thought 
and  reacted  by  evacuating  his  men  to  safety  all  without  ever  realizing  his  own  thought 
process. 14  We  will  see  how  this  lack  of  conscious  recognition  figures  when  applying  the 
aspects  of  intuitive  decision  making  to  future  training,  but  for  now  the  focus  will  remain 
on  how  decisions  are  made. 

Boyd’s  OODA  Loop 

Klein  offers  one  perceptual  model  of  how  decisions  are  made,  but  his  model  fails 
to  illustrate  how  the  connections  and  decisions  are  actually  made.  A  fuller  and  more 
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robust  model  of  the  decision  process  comes  from  John  Boyd  and  his  famous  OODA  loop. 
OODA  stands  for  the  four  phases  of  the  loop:  observe,  orient,  decide,  and  act.  The 
OODA  loop  is  a  mnemonic  for  what  Boyd  believed  were  the  four  phases  of  decision¬ 
making.  Figure  2  in  Appendix  B  illustrates  the  complexity  of  each  step  of  the  loop  with 
each  phase  connected  through  feedback  to  the  ones  that  came  before. 

The  basic  understanding  of  the  OODA  loop  is  that  it  is  a  decision-making  process, 
and  that  the  faster  one  can  use  this  process  relative  to  an  enemy,  the  more  advantage  one 
gains  over  that  enemy  pilot.  But  this  is  an  incomplete  orientation  to  the  OODA  loop. 

The  OODA  loop  is  much  less  a  model  of  decision-making  and  much  more  of  a  model  of 
learning  and  adaptation  for  both  individuals  and,  in  an  extended  sense,  organizations. 
Decisions  are  made,  but  they  are  made  in  the  context  of  the  situation  and  not  in  isolation. 
Examining  Boyd’s  OODA  loop  while  keeping  in  mind  Klein’s  Sources  of  Power 
provides  additional  insight  into  how  people  make  decisions  and  interact  with  the  world. 

The  first  step  of  the  OODA  loop  is  observation.  Observation  is  the  initial  input. 
Boyd  included  unfolding  circumstances,  outside  information,  and  unfolding  interaction 
with  the  environment  as  inputs  into  the  observation  phase.  The  last  of  these  is  important 
enough  to  expand  on  briefly.  The  unfolding  interaction  with  the  environment  is 
recognition  that  observing  a  system  impacts  the  system.  From  Boyd’s  article 
“Destruction  and  Creation15”  we  know  that  he  based  this  concept  on  the  Heisenberg 
Uncertainty  Principle  that  stated  you  could  not  know  both  the  speed  and  location  of  a 
particle  at  the  same  time  as  well  as  recognizing  that  observation  of  a  system  impacted  the 
system.  To  use  a  more  visual  analogy,  using  a  thermometer  to  measure  the  temperature 
of  a  cup  of  coffee  will  lower  the  temperature.  While  measuring,  some  of  the  liquid’s  heat 
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warms  the  thermometer,  and  thus  the  temperature  reported  diminishes.  At  nonnal 
temperatures  this  impact  might  be  small  and  not  significant  for  the  needs  of  the 
measurement,  but  if  the  thermometer  had  been  measuring  the  temperature  of  a  freezer 
immediately  before  the  coffee  measurement,  the  temperature  difference  could  be  more 
dramatic.  The  impact  of  the  observer  on  the  system  is  also  where  the  feedback  from 
decisions  returns  to  restart  the  loop.  The  extent  that  Klein’s  attributes  of  the  expert  apply 
to  the  observe  phase  are  that  experts  making  decisions  would  have  better  foreknowledge 
of  the  input  required  to  make  a  good  decision  and  would  ensure  that  those  inputs  are 
observed  prior  to  moving  into  the  orientation  phase.  The  temptation  would  be  to  use  the 
aspects  of  Klein’s  expertise  to  describe  how  people  observe  the  situation,  but  seeing 
things  that  are  invisible  to  novices  and  seeing  patterns,  anomalies,  and  events  that  did  not 
happen  or  may  soon  happen  are  all  actually  built  from  experience  and  they  are  actually 
part  of  the  second  phase  of  the  OODA  loop,  orientation. 

The  orientation  phase  is  the  most  important  in  the  OODA  loop.  The  sources  of 
power  for  expert  decision-making  all  exist  in  the  orientation  phase  as  well.  Klein’s 
attributes  of  expertise  primarily  result  from  pattern  recognition,  which  is  a  simple  way  of 
explaining  how  people  orient  to  a  situation.  A  majority  of  the  attributes  of  an  expert  are 
applied  in  the  previous  experience  bubble  within  the  orientation  phase,  but  Boyd 
recognizes  that  there  are  other  influences  included  in  orientation  as  well.  He  includes 
new  information,  novel  items  not  previously  experienced,  as  well  as  genetic  and  cultural 
filters.  He  also  includes  analysis  and  synthesis.  Analysis  is  defined  as  the  breaking  down 
of  something  into  pieces  that  may  be  easier  to  understand.  Synthesis  is  the  recombination 
of  these  broken  down  components  into  something  new.  People  use  all  of  these  things  to 
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create  a  perceptual  model  time  after  time. 

Prescriptive  Procedures 

Expert  decision  makers  need  to  deliberately  practice  the  skills  required  to  make 
sound  decisions  within  their  field  of  expertise  and  to  do  so  often.  They  need  to 
repeatedly  practice  the  precise  skills  required  to  create  the  decision-making  experiences 
needed  to  orient  the  problem  correctly  and  analyze  and  synthesize  the  patterns  into 
applicable  forms.  So,  by  understanding  how  people  make  decisions,  we  can  see  the 
immediate  value  of  experience  in  the  decision  making  process,  but  the  experience  must 
be  specific  to  the  area  of  the  decision.  If  the  experience  is  too  far  removed  from  the 
decision,  then  the  decision  maker  will  attribute  the  decision  to  a  novel  situation.  A 
traditional  view  of  the  solution  to  this  problem  is  to  teach  tactics,  techniques,  and 
procedures  (TTPs).  The  assumption  is  that  through  these  TTPs  enough  experience  will 
be  generated  that  eventually  a  pilot  will  be  capable  of  making  decisions,  but  the  TTPs 
remove  the  requirement  for  decision  by  prescribing  standardized  solutions  instead  of 
teaching  the  novice  pilot  how  to  recognize  the  pattern  and  why  the  TTP  is  the  solution  to 
the  problem.  One  example  of  this  disconnect  is  learning  to  ride  a  bicycle  by  using 
training  wheels.  People  do  not  use  the  training  wheels  so  well  that  they  become  an 
ingrained  part  of  the  bicycle  riding  experience;  they  outgrew  the  need  for  training  wheels. 
“Presenting  procedures  to  trainees  gives  them  a  false  sense  of  progress.  This  confidence 
dissipates  when  novices  realize  that  applying  the  procedures  depends  on  the  context  and 
no  one  can  tell  them  what  the  context  is.”16  To  return  to  our  landing  aircraft  example,  the 
procedures  for  student  pilots  list  a  series  of  gates,  a  combination  of  altitude  and  airspeed, 
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which  students  should  hit  when  approaching  for  a  landing.  Like  the  training  wheels  in 
the  previous  example,  the  gates  are  irrelevant  to  the  actual  landing  requirements.  The 
gates  provide  descriptive  characteristics  of  one  possible  safe  landing  glideslope  and 
closure  rate.  The  evaluation  criteria  as  well  as  the  procedures  taught  focus  on  meeting 
the  gates  and  not  what  the  gates  represent.  Understanding  of  closure  rates  and 
glideslopes  is  expected  to  come  when  the  landing  process  has  been  repeated  enough 
times,  but  there  is  no  discussion  of  what  experienced  pilots  are  seeing  and  doing  versus 
the  novices.  Gary  Klein  solved  this  dilemma  through  a  process  known  as  CTA. 

Cognitive  Task  Analysis:  A  Tool  to  Bridge  the  Gap  from  Theory  to  Application 

CTA  is  a  tool  that  attempts  to  translate  how  experts  think  into  strategies  for 

perceiving,  or  orienting  the  situation.  CTA  is  the  description  of  the  expertise  needed  to 

perform  complex  tasks.  It  requires  locating  sources  of  expertise,  evaluating  the  quality  of 

the  expertise,  performing  knowledge  elicitation,  processing  the  findings,  and  applying  the 

findings.17  Traditional  task  analysis  focuses  on  procedures  used,  but  CTA  moves  beyond 

that  into  the  perception,  judgment,  and  decision-making  skills  of  the  experts  in  a  field. 

Step  one  of  CTA  focuses  on  identifying  sources  of  expertise.  CTA  focuses  on  the 

expertise  and  not  the  experts  themselves.  The  emphasis  is  to  “find  individuals  whose 

1  8 

expertise  is  respected  in  the  organization,  in  order  to  learn  how  they  see  their  job.”  Step 
two  is  to  assay  the  knowledge  and  detennine  its  value  and  the  cost  to  extract  it.  Not 
every  piece  of  knowledge  is  worth  the  cost  and  time  to  extract,  so  step  two  allows  for 
weeding  out  some  data  to  focus  the  research.  Step  three  is  to  extract  the  knowledge. 

This  can  be  accomplished  in  a  variety  of  ways  including  structured  interviews, 
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interviews,  or  stories,  about  actual  challenging  events,  interviews  about  concepts  used  to 
think  about  a  task,  and  simulated  tasks  that  require  the  expert  to  think  aloud  during 
performance  or  respond  to  questions  after  the  event.  Step  four  is  to  codify  the 
knowledge,  sort  and  organize  it  to  make  understand  what  it  means.  Understanding  the 
decision  requirements  as  well  as  what  makes  the  decision  difficult  can  allow  trainees  to 
apply  the  information  in  more  practical  ways.  The  final  step  is  to  apply  the  knowledge. 19 
The  key  cues  for  problem  diagnosis  have  been  identified;  now  they  just  need  to  be  taught 
and  applied.  By  understanding  the  expert  perceptions  and  how  they  perceive  the  task, 
relevant  training  can  be  created  to  teach  novices  to  think  like  the  experts.  Once  the 
proper  training  exists,  dedicated  practice  and  repetition  can  enable  the  possibility  of 
expertise. 

CTA  is  a  tool  that  may  be  utilized  to  understand  how  experts  think  about  the  tasks 
they  perform  and  provide  insight  into  how  to  train  less  experienced  people  to  think  like 
the  experts.  The  video  game  Space  Fortress  and  its  impact  on  the  Israeli  Air  Force  show 
the  significant  impact  of  training  novice  pilots  in  skills  that  are  the  most  relevant  to  their 
success.  Space  Fortress  has  moved  beyond  aviation  to  basketball.  The  2006  NCAA 
champion  Florida  Gators  used  it.20  The  improvements  of  memory  and  attention  appear  to 
benefit  any  field  that  requires  numerous,  rapid,  strategic  decisions.  A  CTA  study  should 
capture  the  most  relevant  skills  needed  to  be  an  expert  pilot,  such  as  those  from  the  Space 
Fortress  game,  and  should  be  conducted  to  capture  the  current  expertise  in  military 
aviation.  Considering  the  last  ten  years  of  war  now  is  a  prime  time  to  find  experts  whose 
actual  combat  experience  could  directly  contribute  to  applicable  future  training  for  Naval 
and  Marine  Corps  Aviators.  The  CTA  may  need  to  be  broken  down  into  general 
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aviation,  fixed  wing  and  helicopter,  and  possibly  as  far  as  specific  aircraft  to  gain  the  full 
benefit. 

Conclusion  and  Recommendations:  Think  Like  an  Expert 

So  far  this  study  explored  the  desired  outcome  of  expert  pilots,  examined  what  it 
means  to  be  an  expert  and  showed  why  expertise  is  beneficial.  It  continued  with  an 
examination  of  judgment  and  how  people  make  decisions  that  drove  the  discussion  back 
into  the  impact  of  experience  on  the  ability  to  make  sound  decisions.  The  US  Navy 
should  use  CTA  to  study  its  expert  pilots  to  enable  training  to  increase  pilots’  beginning 
capability  for  effective  intuitive  decision-making.  A  professional  analysis  of  how  expert 
pilots  think  about  their  skills  and  decisions  will  significantly  enhance  the  naval  aviation 
community  understanding.  The  argument  concludes  that  effective  intuitive  decision¬ 
making  is  the  primary  attribute  of  expert  pilots,  but  this  claim  might  make  some  people 
uncomfortable.  Intuitive  decision-making  implies  that  intuition  provides  the  best  answer 
to  most  flying  choices  instead  of  rational  calculation  and  by-the-book  knowledge. 
Appendix  C  addresses  the  risks  of  intuition  and  how  to  mitigate  those  risks. 

Unfortunately,  no  data  exists  to  accurately  predict  the  results  of  implementing  a 
training  program  based  on  this  theory.  The  theories  are  all  based  on  scientific  study,  but 
their  focus  was  on  understanding  how  people  think,  make  decisions,  or  become  experts, 
and  is  not  focused  on  developing  an  educational  methodology  that  can  be  tested  to 
improve  future  performance.  The  synthesis  of  these  theories  provides  a  foundational 
theory  for  future  application,  but  testing  is  still  required  to  prove  the  theory’s  soundness 
and  demonstrate  a  numerical  advantage. 
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The  anticipated  advantages  are  less  tangible  than  an  immediate  reduction  in  flight 
hours  or  training  time,  although  those  benefits  may  occur  as  well.  The  primary 
expectation  is  that  a  shift  in  training  methodology  will  bridge  the  gap  between  the  end  of 
traditional  flight  school  and  when  a  pilot  becomes  an  expert.  Many  liberal  arts  colleges 
claim  that  they  do  not  teach  students  what  to  think,  but  rather  how  to  think.  They  see  the 
benefit  in  education  not  just  in  the  accumulation  of  knowledge,  but  also  in  learning  the 
skills  needed  to  gain  future  knowledge.  By  providing  additional  training  for  the 
instructors  in  flight  school  to  enable  them  to  teach  novices  about  how  the  experts  think, 
the  expectation  is  that  the  novices  will  not  only  learn  the  basics  of  flying  but  also  the 
techniques  needed  to  continue  to  improve  after  they  leave  flight  school.  By  learning  how 
to  learn  like  an  expert,  the  pilot  increases  his  ability  to  train  effectively  toward  expertise. 

Application  of  these  recommendations  requires  a  CTA  study  of  expert  Naval 
pilots  followed  by  improved  training  of  the  instructors  to  teach  them  better  techniques  for 
explaining  procedures  to  the  student  pilots.  No  additional  student  training  time  is 
anticipated.  In  fact,  if  the  Space  Fortress  game  is  any  indication  of  possible  effectiveness 
of  using  the  correct  techniques,  training  time  might  actually  drop.  Whatever  the  student 
training  time  becomes,  the  expectation  is  that  the  students  who  graduate  would  be  better 
prepared  to  harness  their  experiences  in  their  operational  squadrons  and  become  expert 
pilots.  There  is  additional  hope  that  the  students  will  benefit  from  the  shift  in  training 
focus  during  flight  school,  but  there  is  currently  no  way  to  determine  if  the  effects  will 
manifest  within  the  flight  school  training. 

The  Navy  needs  expert  pilots  who  can  react  quickly  and  intuitively  in  complex 
situations  to  execute  sound  judgment.  Expertise  provides  unique  capabilities  of  pattern 
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recognition  and  mental  simulation  that  enable  accurate  intuitive  decision-making. 
Accurate  intuitive  decision-making  is  necessary  to  have  sound  judgment,  especially  in 
time  critical  situations  such  as  aviation.  Intuitive  decision-making  relies  on  experience 
and  expertise,  so  training  specific  skills  over  time  builds  the  foundation.  CTA  is  a  tool 
that  is  available  to  decipher  the  exact  skills  needed  for  dedicated  practice  to  create 
expertise.  Based  on  the  skills  needed  and  an  understanding  of  the  intuitive  decision 
making  process,  improvements  can  be  made  to  the  flight  training  program  that  should 
improve  the  ability  to  create  and  train  expert  pilots. 
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APPENDIX  A:  The  Expert  Advantage 

To  understand  the  goal  of  creating  expert  pilots,  it  is  essential  to  understand 
expertise,  what  it  is,  and  is  not,  and  why  it  is  desired.  Gary  Klein  explains  in  his  book 
Sources  of  Power  that  “One  view  of  experts  is  that  they  have  accumulated  lots  of 
knowledge. ”  While  this  view  is  correct  in  the  basic  concept  that  experts  have  more 
knowledge,  it  creates  a  false  image.  Experts  have  not  learned  all  there  is  to  leam  even 
within  their  field,  they  just  understand  the  field  well  enough  to  leam  differently  than  a 
novice.  If  the  knowledge  of  an  expert  were  pictured  as  a  library,  the  vastness  of  the 
library  alone  is  not  enough.  The  knowledge  must  be  organized  to  provide  easy  access  to 
the  vast  stores.  This  ability  to  organize  the  knowledge  available  enables  experts  to 
effectively  see  things  that  are  invisible  to  the  novice.  Experts  are  able  to  see  patterns  that 
novices  do  not  notice,  anomalies,  events  that  did  not  happen  and  other  violations  of 
expectancies,  the  big  picture  (situational  awareness),  the  way  things  work,  opportunities 
and  improvisations,  events  that  either  already  happened  (the  past)  or  are  going  to  happen 

(the  future),  differences  that  are  too  small  for  novices  to  detect,  and  their  own  limitations. 

22 

All  of  these  aspects  can  be  derived  from  pattern  matching  and  mental  simulation. 

Pattern  matching  applies  to  being  able  to  see  patterns  that  novices  do  not  notice  as 
well  as  seeing  anomalies.  Novices  cannot  see  relationships  that  are  obvious  to  experts. 
For  example,  an  expert  fireman  commander  can  look  at  a  burning  building  and  envision 
the  structure,  such  as  stairs,  elevators,  and  support  beams,  and  can  envision  what  is 
happening  inside  the  building.  Pattern  recognition  gives  one  expert  fireman  enough 
subconscious  situational  awareness  that  he  is  able  to  evacuate  a  building  just  before  the 
floor  collapses.  Because  there  were  anomalies  to  the  pattern  of  the  fire,  the  fireman  felt 
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that  there  was  something  different  or  wrong.  The  fire  was  not  progressing  the  way  it 
should  based  on  where  it  was  thought  to  be.  The  anomalies  were  so  subtle  that  they  were 
not  consciously  recognized,  but  the  fireman’s  expertise  allowed  him  to  register  the 
anomalies  subconsciously  and  react  to  the  pattern  he  recognized  intuitively.  This 
ability  of  experts  to  see  the  pattern  enables  their  overall  ability  to  see  the  big  picture  or 
maintain  situational  awareness.  A  novice  is  much  more  likely  to  get  bogged  down  in  the 
details  of  the  situation  and  lose  sight  of  the  big  picture."  Pattern  matching  enables  an 
expert  to  analyze  a  situation  into  parts  that  are  easier  to  visualize  and  track. 

Mental  simulation  allows  experts  to  see  inside  events  and  objects  and  understand 
the  way  things  work.  They  have  mental  models  of  how  things  are  supposed  to  be  done 
and  can  quickly  see  if  they  are  not  going  as  expected.  These  mental  models  also  enable 
the  expert  to  take  advantage  of  or  create  opportunities  or  improvisations."  In  military 
aviation,  these  opportunities  are  essential  to  be  able  to  recognize  and  exploit  an  enemy 
mistake  or  utilize  an  advantage.  By  recognizing  the  pattern  and  being  able  to  mentally 
model  what  is  happening,  experts  see  opportunities  that  a  novice  pilot  would  not 
recognize.  The  ability  to  anticipate  the  past  and  future  from  a  snapshot  is  also  directly 
related  to  the  expert’s  ability  to  model  the  situation  mentally.  This  ability  is  also 
fundamental  in  the  ability  to  look  at  the  world  through  someone  else’s  eyes.  Being  able 
to  reorient  to  another  person’s  perspective  gives  a  significant  advantage  when  in  an 
adversarial  engagement  such  as  an  aerial  dogfight.  If  you  can  put  yourself  in  your 

77 

opponent’s  place,  you  can  anticipate  what  he  might  do  next  based  on  what  he  perceives." 

The  ability  to  make  fine  discriminations  and  the  ability  to  manage  one’s  own 
limitations  stem  from  an  increase  in  the  situational  awareness  that  increases  as  someone 
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gains  expertise.  The  exact  method  of  this  increase  is  not  fully  understood,  but  the  effects 

28 

are  clear." 

What  this  means  in  terms  of  naval  aviation  is  that  there  are  significant  concrete 
advantages  to  having  expert  pilots.  The  expert  pilots  will  be  able  to  recognize  patterns 
and  pick  up  on  anomalies  that  will  increase  the  safety  of  the  flight.  The  ability  to 
organize  and  process  the  relevant  data  enables  them  to  make  sound  decisions  more 
quickly  than  a  novice.  The  ability  to  reorient  to  an  adversary’s  perspective  enables  the 
expert  pilot  to  predict  the  adversary’s  behavior  and  anticipate  and  exploit  opportunities. 

Rather  than  expertise  being  the  accumulation  of  more  and  more  knowledge, 
expertise  is  learning  how  to  perceive  more;  the  knowledge  and  rules  are  incidental.  By 
focusing  on  how  to  build  and  recognize  patterns  instead  of  focusing  on  procedures  to 
follow,  people  can  be  trained  to  achieve  expertise  more  quickly.  Rules,  facts,  and  tactics, 
techniques,  and  procedures  (TTPs),  vice  intuition,  have  previously  dominated  the  efforts 
in  the  field  of  technical  training.  Skills  and  knowledge  are  seen  as  a  set  of  procedures  to 
be  deconstructed  and  taught  systematically.  This  strategy  is  effective  for  simple, 
procedural  tasks,  but  they  are  not  designed  to  gain  higher  levels  of  expertise  or  judgment. 
Teaching  procedures  gives  novices  a  false  sense  of  confidence  that  dissipates  when  the 
novice  realizes  that  the  procedure  is  still  dependent  on  the  context  in  which  it  is  applied 
and  the  context  cannot  be  taught.  Judgment  and  decisions  are  rarely  straightforward  and 
if  the  goal  is  to  teach  people  to  think  like  experts  we  must  understand  how  the  experts  are 
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thinking,  including  their  strategies  and  perceptions." 
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APPENDIX  B:  FIGURES 
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APPENDIX  C:  The  Risks  of  Intuition  Addressed 

While  intuition  is  based  on  experience,  experience  does  not  imply  that  intuition  is 
infallible  and  addressing  the  common  illusions  of  intuition  is  essential  for  mitigating 
them  in  training  designed  to  improve  intuitive  decision-making.  Three  everyday  illusions 
of  intuition  are  the  illusions  of  attention,  memory,  and  confidence. 

The  human  mind  has  an  incredible  capacity  to  focus  on  essential  tasks  while 
ignoring  all  irrelevant  information.  This  capability  is  also  the  source  of  the  illusion  of 
attention.  In  a  psychology  experiment  in  the  1970’s  a  video  is  shown  of  two  teams  of 
basketball  players  passing  the  basketball.  One  team  is  in  white  and  the  other  black.  An 
observer  is  asked  to  count  the  number  of  passes  made  by  the  white  team.  Halfway 
through  the  video  a  girl  in  a  full  body  gorilla  suit  walked  into  the  middle  of  the  scene, 
stopped  in  the  middle  of  the  players,  faced  the  camera,  thumped  her  chest,  and  then 
walked  off  after  spending  nine  seconds  on  the  screen.  Roughly  half  of  the  observers 
never  saw  the  gorilla.  The  invisible  gorilla,  as  the  experiment  has  become  known,  is  a 
primary  example  of  the  illusion  of  attention.  The  illusion  is  that  people  experience  far 
less  of  the  visual  world  than  they  think  they  do.  Similar  experiments  exist  of  pilots  in 
simulators  focusing  on  a  task  such  as  using  a  head-up  display  (HUD)  and  missing  an 
airliner  pulling  onto  the  runway  where  they  are  landing.  The  more  mentally  complicated 
the  focused  task  the  higher  the  percentage  of  people  who  experience  the  illusion  of 
attention.  In  other  words,  the  more  complicated  the  task  the  more  attention  people  focus 
on  it  and  the  less  attention  is  available  to  notice  something  unexpected.  For  pilots  this 
would  be  described  as  a  loss  of  situational  awareness.  The  solution  to  the  dilemma  of  the 
invisible  gorilla  is  expertise.  Experienced  basketball  players  are  much  more  likely  to 
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notice  the  gorilla  in  the  basketball-passing  video,  but  team  handball  players  are  not. 
Expertise  helps  you  notice  the  unexpected,  but  only  within  the  context  of  the  expertise. 
As  Gary  Klein  covered  in  his  attributes  of  expertise,  one  of  the  qualities  of  an  expert  is 
the  ability  to  see  the  invisible,  in  this  case  see  the  invisible  gorilla.  To  teach  this  ability 
requires  that  attention  be  drawn  to  unexpected  objects  that  are  missed. 

The  illusion  of  memory  is  defined  as  the  disconnection  between  how  people  think 
memory  works  and  how  it  actually  works.  People  believe  that  memory  is  immutable, 
acting  as  a  permanent  and  accurate  recording  of  what  happened,  but  in  reality  memory 
depends  on  a  combination  of  what  actually  happened  as  well  as  how  one  made  sense  of 
what  happened  in  context  of  all  experience.  This  reservoir  of  experience  is  constantly  in 
flux.  The  mind  is  quite  capable  of  filling  in  the  blanks  in  a  memory  to  what  one  thinks 
should  be  there.  For  example,  if  someone  saw  a  picture  of  an  office  with  desks,  chairs, 
shelves,  and  other  office  equipment  and  were  asked  later  to  recall  the  image  there  is  a 
good  chance  that  books  and  filing  cabinets  not  in  the  image  would  exist  in  the  recall 
because  by  defining  the  image  as  an  office  the  items  should  have  been  in  the  image  and 
their  absence  is  incongruent  with  the  way  people  make  sense  of  the  office  image.  34 

The  most  relevant  issue  of  the  illusion  of  memory  to  flight  training  is  that  so 
much  of  the  training  provided  comes  from  explanations  of  the  flight  instructors  who  have 
enough  experience  that  their  skills  have  become  intuitive  and  they  no  longer  truly 
understand  what  techniques  they  use  to  be  so  effective.  The  CTA  is  an  unbiased  method 
for  an  outside  observer  to  extract  the  truth  from  the  memories  and  skills  of  the  expert 
pilots.  The  analysis  will  then  provide  a  valuable  new  language  to  instruct  the  novice 
pilots.  The  risk  is  that  so  much  of  CTA  comes  from  the  memories  of  the  expert  pilots 
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themselves  through  interviews  and  stories.  The  use  of  memories  must  be  combined  with 
the  use  of  other  techniques  such  as  observing  the  pilots  while  watching  videos  or 
operating  simulators  to  track  eye  movements  and  verify  key  inputs  into  their  decision 
process.  By  combining  the  experiences  of  enough  expert  pilots  with  experimental  data 
analysis  on  relevant  inputs,  the  CTA  will  be  much  more  likely  to  produce  accurate 
perceptions  and  skills  necessary  for  success. 

The  final  everyday  illusion  to  be  discussed  is  the  illusion  of  confidence.  The 
illusion  of  confidence  comes  in  two  forms.  The  first  is  that  people  who  appear  confident 
are  trusted  more,  and,  second,  people  believe  they  are  underrated.  The  illusion  causes 
people  to  overestimate  their  own  qualities  especially  relative  to  other  people  as  well  as 
judge  competence  based  on  the  appearance  of  confidence.  The  caveat  to  this  illusion  is 
inversely  proportional  to  the  skill  of  the  individual,  so  the  least  skilled  are  most  likely  to 
be  overconfident.  People  tend  to  think  that  good  performances  reflect  superior  abilities 
while  excusing  poor  performances  as  a  result  of  circumstances  outside  ones  control. 
Teaching  someone  how  to  improve  on  a  task,  however,  has  proven  to  significantly  reduce 
overconfidence.  Making  people  more  competent  increases  their  ability  to  judge  their 
own  competence.  Incompetence  causing  overconfidence  tells  us  that  study  and  practice 
of  a  task  allows  people  to  improve  in  the  task  as  well  as  knowing  how  well  they  perform 
the  task.  When  people  learn  new  skills  their  skill  level  is  low  and  confidence  is  higher 
than  it  should  be.  As  their  skills  improve  their  confidence  also  rises,  but  at  a  slower  rate 
than  their  skill,  so  eventually  the  skill  level  and  confidence  match.  ~  Awareness  of  this 
illusion  is  key  to  understanding  the  associated  risks.  Flight  training  is  inherently 
dangerous  and  becomes  more  so  when  students  and  instructors  are  unaware  of  some  of 
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the  risks.  Overconfidence  is  not  an  unknown  risk,  but  targeting  specific  skills  needed  to 
create  expertise  can  build  the  experience  necessary  to  improve  skills  and  reduce  the  risk 
of  overconfidence. 


1  F.  Brecke,  “Instructional  Design  for  Aircrew  Judgment  Training,”  (Oct  1982,  145-160), 
www.faa.gov/library/online  libraries/aerospace. ../sd/.../brecke  f.pdf  (accessed  Oct  28, 
2011),  145. 

2  K.  Anders  Ericsson,  Michael  J.  Prietula,  and  Edward  T.  Cokely,  “The  Making  of  an 
Expert,”  Harvard  Business  Review  (July- August  2007), 

http://www.google.com/url?sa=t&rct=i&q=&esrc=s&source=web&cd=T&ved=OCCYQF 

jAA&url=http%3A%2F%2F  14 1.14.1 65 .6%2Fusers%2Fcokelv%2FEricsson  Preitula  % 

26  Cokely  2007  HBR.pdf&ei=UAsiT6vKOOitOeHe7KHiCA&usg=AFOiCNHRiOgpf 

H5udVli30cRPQKZF2DT0A  (accessed  Jan  27,  2012),  2. 

3  Ericsson  et  al.  1 . 

4  Ericsson  et  al.  3. 

5  Geoff  Colvin,  Taierit  is  Overrated:  What  Really  Separates  World-Class  Performers 
from  Everybody  Else  (NY:  Portfolio,  2010),  66. 

6  Colvin.  67-70. 

7  Colvin.  70-72. 

8  Sian  Beilock,  Choke:  What  the  Secrets  of  the  Brain  Reveal  about  Getting  It  Right  Wlren 
You  Have  To  (NY:  Free  Press,  2010),  86-88. 

9  http://dictionary.reference.com/browse/judgment 

10  Brecke.  147-152. 

11  Klein.  260-264. 

12  Robert  Jervis,  System  Effects:  Complexity  in  Political  and  Social  Life  (Princeton,  New 
Jersey:  Princeton  University  Press,  1997),  12-16. 

13  Klein.  31. 

14  Klein.  1-35. 

15  John  Boyd,  “Destruction  and  Creation,”  unpublished  essay  (Septemberl976) 
www.goalsvs.com/books/documents/DESTRUCTION  AND  CREATION.pdf  (accessed 
Jan  30,  2012),  1-8. 

16  Klein.  169. 

17  Klein.  169-170. 

18  Klein.  171. 

19  Klein.  171-175. 

20  Beilock.  86-88. 

21  Gary  Klein,  Sources  of  Power:  How  People  Make  Decisions  (MA:  The  MIT  Press, 
1999),  147. 

22  Klein.  148-149. 

23  Klein.  149-150. 

24  Klein.  31-33. 


D 


Tempest  -  Building  Potential  for  Pilot  Expertise 


25  Klein.  152. 

26  Klein.  152-154. 

27  Klein.  155-157. 

28  Klein.  157-161. 

29  Klein.  168-169. 

30  Klein.  289. 

31  http://openlearn.open.ac.uk/mod/oucontent/view.php?id=403128&section=3.2 

32  Frans  P.  B.  Osinga,  Science,  Strategy  and  War:  The  Strategic  Theory  of  John  Boyd 
(NY:  Routledge,  2007),  231. 

33  Christopher  Chabris  and  Daniel  Simons,  The  Invisible  Gorilla:  And  Other  Ways  Our 
Intuitions  Deceive  Us  (NY:  Crown,  2010),  1-42. 

34  Chabris  et  al.  43-79 

35  Chabris  et  al.  80-115. 


E 


